Dynamic Data Rectification Using the Expectation Maximization Algorithm

Although on-line measurements play a ®ital role in process control and monitoring ( process performance, they are corrupted by noise and occasional outliers such as noise ) spikes . Thus, there is a need to rectify the data by remo®ing outliers and reducing noise effects. Well-known techniques such as Kalman Filtering ha®e been used effecti®ely to filter noise measurements, but it is not designed to automatically remo®e outliers. A new methodology based on the Kalman filter rectifies noise as well as outliers in measurements. Filter equations were formulated in the form of probability distributions. Then the Expectation-Maximization algorithm was used to find the maximum-likelihood estimates of the true measurement ®alues based on a state-space model, past data, and current obser®ations. This approach was e®aluated when the assumption of normally distributed outliers is not ®alid. The method can be used with any dynamic process model, as shown by integrating it with an extended Kalman Filter and by an augmented linear state-space model to account for unmeasured disturbances. It also can be used to pro®ide diagnostic information about changes to the process or sensor failures.